Minimizing Testing Overheads in Database Migration Lifecycle

As part of their information management lifecycle, organizations periodically face the important and inevitable task of migrating databases from one software and hardware platform to another. Apart from ensuring accuracy and consistency, information managers of such organizations need to minimize the costs associated with the database migration process. Some of the migration tasks such as identification of databasedependent applications and the process of data replication are primarily governed by the technology used and have relatively predictable costs. On the other hand, the cost of testing and verification in the migration life-cycle depends significantly on the planning and dominates other costs. We refer to this problem of optimizing testing cost as Database Migration Problem (DBMP). DBMP is challenging for enterprises because large number of databases and applications are affected and a variety of constraints force the migration to spread over multiple phases (or migration waves, as they are typically called in the industry parlance). After each migration wave, significant overheads arise for ensuring data integrity and testing the applications for correctness. We ∗ Corresponding author. Email: [email protected] † Currently at Chennai Mathematical Institute, India. Email: [email protected] ‡Currently at Nanyang Technological University, Singapore. Email: [email protected] §Dept. of Mathematics, Massachusetts Inst. of Tech., USA. Email: [email protected] ¶ Currently unaffiliated with TCS. Email: [email protected] ‖ Currently unaffiliated with TCS. Email: [email protected] International Conference on Management of Data COMAD 2010, Nagpur, India, December 8–10, 2010 c ©Computer Society of India, 2010 have observed that the industry practice for migration planning is based on the experience and intuition of a few software architects and administrators. This often results in delayed migration schedules and spiraling costs. However, a careful partitioning of databases into waves can lower the testing overheads and result in significant financial savings of several hundred thousand dollars. Surprisingly we did not find any literature that addresses this problem either. Hence, in this paper, we focus on minimizing the testing overheads of the data migration life-cycle. We begin by showing that DBMP is NP-hard. Then, based on our real-life experience, we formulate DBMP so that it is amenable to formal, rigorous analysis and provide algorithms that cater to different scenarios, likely in practice. For small problem instances, we provide an optimal solution using integer linear programming. For larger instances, we formulate DBMP as a hyper-graph partitioning problem and use the wellknown hMETIS tool for solving it. hMETIS provides good solutions quickly, but violates some of the constraints that are important in industry, for instance the total size of databases packed in a wave cannot exceed the maximum wave size limit. Finally, to tackle such scenarios, we propose a new algorithm WAVE-FIT. Using experimental evaluation, we show that WAVEFIT provides solutions comparable to hMETIS, in reasonable time without violating any constraint.